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Abstract 



A general maximum principle (necessary and sufficient conditions) 
for an optimal control problem governed by a stochastic differential 
equation driven by an infinite dimensional martingale is established. 
The solution of this equation takes its values in a separable Hilbert space 
and the control domain need not be convex. The result is obtained by 
using the adjoint backward stochastic differential equation. 
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1 Introduction 

This paper studies the following form of a controlled stochastic differential 
equation (SDE in short): 



where M is a continuous martingale taking its values in a separable Hilbert 
space K, while F, G are some mappings with properties to be given later and 
u{-) represents a control variable. We will be interested in minimizing the cost 
functional: 
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dX{t) = F{X{t), u{t))dt + G{X(t))dM{t), 0<t<T, 
X(0) = xo. 




Jiu{-))=E[ [ t{X<-\t),u{t))dt + h{X<\T))] 
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over a set of admissible controls. 

We shall follow mainly the ideas of Bensoussan in [10], [11], Zhou in [37], 
[36] , 0ksendal et al. [26], and our earlier work [1]. The reader can see our 
main results in Theorems fl5.3[ 16. ip . 

We recall that SDEs driven by martingales are studied in [16], [21], [31], [15] 
and [6] . In fact in [6] we derived the maximum principle (necessary conditions) 
for optimality of stochastic systems governed by SPDEs. However, the results 
there show the maximum principle in its local form and also the control domain 
is assumed to be convex. In this paper we shall try to avoid such conditions 
as we shall shortly talk about it. Due to the fact that we are dealing here 
with a non-convex domain of controls, it is not obvious how one can allow the 
control variable u{t) to enter in the mapping G in (11. ip and obtain a result like 
Lemma 14.41 This issue was raised also in [10]. Nevertheless, in some special 
cases (see [8]) we can allow G to depend on the control, still overcome this 
difficulty, and prove the maximum principle. The general result is still open 
as pointed out in [6l Remark 6.4]. 

The maximum principle in infinite dimensions started after the work of 
Pontryagin [30]. The reader can find a detailed description of these aspects 
in Li & Yong [22] and the references therein. An expanded discussion on the 
history of maximum principle can be found in [361 P- 153-156]. On the other 
hand, the use of (linear) backward stochastic differential equations (BSDEs) 
for deriving the maximum principle for forward controlled stochastic equations 
was done by Bismut in [12]. In this respect, one can see also the works of 
Bensoussan in [10] and [11]. In 1990 Pardoux & Peng, [27], initiated the 
theory of nonlinear BSDEs, and then Peng studied the stochastic maximum 
principle in [28] and [29]. Since then several works appeared consequently 
on the maximum principle and its relationship with BSDEs. For example 
one can see [17], [18], [19], [33] and [36] and the references of Zhou cited 
therein. Our earlier work in p] has now opened the way to study BSDEs 
and backward SPDEs that are driven by martingales. One can see [23] for 
financial applications of BSDEs driven by martingales, and [9], [20], [H] and 
[7] for other applications. 

In this paper the convexity assumption on the control domain is not re- 
quired, as we shall consider a suitable perturbation of an optimal control by 
means of the spike variation method in order to derive the maximum principle 
in its global form. Then we shall provide sufficient conditions for optimality of 
our control problem. The results will be achieved mainly by using the adjoint 
equation of (II. ip . which is a BSDE driven by the martingale M. This can be 
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seen from equation (15 .2^ in Section [5l It is quite important to realize that the 
adjoint equations of such SDEs are in general BSDEs driven by martingales. 
This happens also even if the martingale M, which is appearing in equation 
f ll.ip . is a Brownian motion with respect to a right continuous filtration be- 
ing larger than its natural filtration. There is a discussion on this issue in 
Bensoussan's lecture note pTOf Section 4.4], and in pQ and its erratum 

The paper is organized as follows. Section [2] is devoted to some prelimi- 
nary notation. In Section [3] we present our main stochastic control problems. 
Then in Section H] we establish many of our necessary estimates, which will be 
needed to derive the maximum principle for the control problem of fll.ip . The 
maximum principle in the sense of Pontryagin for the above control problem 
is derived in Section |5l In Section |6] we establish some sufficient conditions for 
optimality for this control problem, and present some examples as well. 

2 Preliminary notation 

Let {Q, J-", P) be a complete probability space, filtered by a continuous filtration 
{J^t}t>o, in the sense that every square integrable ii'- valued martingale with 
respect to {J-'t , < t < T} has a continuous version. 

Denoting by V the predictable a- algebra of subsets of f2 x [0,T] we say 
that a i^'- valued process is predictable if it is V/B{K) measurable. Suppose 
that A4'^Qrp^{K) is the Hilbert space of cadlag square integrable martingales 
{M(t),0 <t< T}, which take their values in K. Let Ai'^Qj,^{K) be the sub- 
space of (K) consisting of all continuous square integrable martingales in 
K. Two elements M and of A^^ rp^{K) are said to be very strongly orthogonal 
(or shortly VSO) if 

E [M(r) (g) N{t)] = E [M(0) ® N{0)], 

for all [0, T] - valued stopping times r. 

Now for M G Ai'^^'^rp-^{K) we shall use the notation < M > to mean the 
predictable quadratic variation of M and similarly << M » means the 
predictable tensor quadratic variation of M, which takes its values in the 
space Li{K) of all nuclear operators on K. Precisely, M M— << M >> G 
A^^g'ipj(Li(_ft')). We shall assume for a given fixed M G Ai'^Q^rp-^{K) that there 
exists a measurable mapping Q(-) : [0,T] x Q ^ Li{K) such that Q(t) is sym- 
metric, positive definite, Q{t) < Q for some positive definite nuclear operator 
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Q on and satisfies the following equality: 

<< M »t = [ Q{s)ds. 
Jo 

We refer the reader to Example 16.21 for a precise computation of this process 

For fixed (t,oj), we denote by LQ(^t,Lo){K) to the set of all linear operators 
: Q^/\t,u){K) K and satisfy i^Q}''^{t,u) e L2{K), where L2{K) is the 
space of all Hilbert-Schmidt operators from K into itself. The inner product 
and norm in L2{K) will be denoted respectively by (■, ■')^ and || ■ ||2- Then the 
stochastic integral ^{s)dM{s) is defined for mappings $ such that for each 
{t,u), ^{t,u) e LQ(_t,uj){K), ^Q^/'^{t,u){h) y h e K is predictable, and 



E[ r\\{^Q'/'){t)\\ldt]<oo. 
Jo 



Such integrands form a Hilbert space with respect to the scalar product 
($i,$2) ^ E [/o^($iQ^/^(t) ,$2Q^/^(t)> dt]. Simple processes taking values 
in L{K;K) are examples of such integrands. By letting A'^{K;V, M) be the 
closure of the set of simple processes in this Hilbert space, it becomes a Hilbert 
subspace. We have also the following isometry property: 



E 



r <^{t)dM{t)\^] = e \ r \ Mt)Q'/\t)\\ids\ (2.1 

Jo ^ Jo 



for mappings $ G A'^{K; V, M). For more details and proofs we refer the reader 
to [25]. 

On the other hand, we emphasize that the process Q(-) will be play an 
important role in deriving the adjoint equation of the SDK (11. ip as it can be 
seen from equations (15. ip . (15. 2p in Section [5l This is due to the fact that the 
integrand $ is not necessarily bounded. More precisely, it is needed in order 
for the mapping VxH, which appear in both equations, to be defined on the 
space L2{K), since the process Z"^^'^ there need not be bounded. This always 
has to be considered when working with BSDEs or BSPDEs driven by infinite 
dimensional martingales. 

Next let us introduce the following space: 

L^O,T;E) := : [0,T] x ^] ^ E, predictable and E [ /" |^(t)prft] < oo }, 

Jo 

where E' is a separable Hilbert space. 
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Since Q{t) < Q for all t G [0, T] a.s., it follows from [3l Proposition 2.2] that 
if $ G L^O, T; Lq{K)) (where as above Lq{K)) = L2{Q^/\K); K)), the space 
of all Hilbert-Schmidt operators from Q^/^{K) into K), then $ G A^{K; V, M) 
and 

E[ f mt)Q''\midt]<E[ f mt)\\i^iK)dt]. (2.2) 

JO JO 

An example of such a mapping $ is the mapping G in equation (II. ip : see the 
domain of G in the introduction of the following section. 



3 Formulation of the control problem 

Let (9 be a separable Hilbert space and [/ be a nonempty subset of O. We 
say that u{-) : [0, T] x C is admissible if u{-) G L^(0,T; C) and u{t) G 

a.e., a.s. The set of admissible controls will be denoted by Uad- 
Let F : K X O ^ K, G : K ^ Lq{K), £ : K x O ^ R and h : K ^ R he 
measurable mappings. Consider the following SDE: 

( dX{t)=F{X{t),u{t))dt+ G{X{t))dM{t), tG[0,T], 

\ XiO) = xoeK. ' 

If assumption (El), which is stated below, holds, then (13.11) attains a unique 
solution in L^{0, T; K). The proof of this fact can be gleaned from [31] or [32] . 
In this case we shall denote the solution of ( 13.11) by X"'^ ^ 

Our assumptions are the following. 
(El) F, G, £, h are continuously Frechet differentiable with respect to x, F 
and £ are continuously Frechet differentiable with respect to m, the derivatives 
Fx, Fu, GxAx,C-u are uniformly bounded, and 

\hx\L{K;K) <k{l + \x\k) 

for some constant A; > 0. 

In particular, \Fx\l(k,k) < Ci, \\Gx\\L{K,LQiK)) < C2, \F^\l(o,k) < C3, for 
some positive constants Ci, i = 1,2, 3, and similarly for i. 
(E2) ix satisfies Lipschitz condition with respect to u uniformly in x. 

Consider now the cost functional: 

J(m(-)) :=E[ r £{X<\t),uit))dt + h{X<\T))], (3.2) 
Jo 
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for u(-) e Uad- 

The control problem here is to minimize fl3.2p over the set Uad- Any «*(■) G 
Uad satisfying 

J{u*{-)) = inf{J(n(-)) : u{-) E Uad} (3.3) 

is called an optimal control, and its corresponding solution X* := X"^ -* to 
(13. ip is called an optimal solution of the stochastic optimal control problem 
(I3.ip -( l3l3|) . In this case the pair {X* ,u*{-)) in this case is called an optimal 
pair. 

Remark 3.1 We mention here that the mappings F,G and i in ^3. ij) and 
^3.2^) can he taken easily to depend on time t with a similar proof as established 
in the following sections, hut rather, having more technical computations. 

Since this control problem has no constraints we shall deal generally with 
progressively measurable controls. However, for the case when there are final 
state constraints, one can mimic our results in Sections dH El [6]), and use 
Ekeland's variational principle in a similar way to [21], |2H] or [36]. 

In the following section we shall begin with some variational method in 
order to derive our main variational inequalities that are necessary to establish 
the main result of Section [5l 



4 Estimates 

Let (X*,u*(-)) be the given optimal pair. Let < to < ^ be fixed and 
< £ < T — to- Let f be a random variable taking its values in [/, J^tQ- 
measurable and sup |f (w)| < oo. Consider the following spike variation of the 

control u*{-): 

^^ ^"It; if tG [to, to + 5]. ^ ^ 

Let denote the solution of the SDE (13. ip corresponding to Me(-)- We 

shall denote it briefiy by X^. Observe that Xs{t) = X*{t) for all < t < to. 

The following lemmas will be very useful in proving the main results of 
Section [51 
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Lemma 4.1 Let (El) hold. Assume that {p{t), to <t <T} is the solution of 
the following linear equation: 



r dp{t) = F,{X*{t),u*{t))p{t)dt + G^{X*{t))p{t)dM{t), to<t<T, 
\ p{to) = FiX*{to),v) - FiX*{to),u*{to)). 



for some positive constant C. 

Proof. With the help of (El) apply Ito's formula to compute |p(i)P, and 
take the expectation. The required result follows then by using Gronwall's 
inequality. ■ 

Lemma 4.2 Assuming (El) we have 



Then 



sup E[\p{t)\'] <C 



E[ sup \XS)-X*{t)\^] = o{e). 



to<t<T 



Proof. For to < t < to + s one observes that 




+ [\F{X*{s),v)-F{X*{s),u*{s))]ds 




to 



(4.3) 



or, in particular, 



X,{t) - X*{t)\'^ <3{t 



- to) f \F{X,{s),v) - F{X*{s),v)\'' ds 



+ 3 (i - to) f \F{X*{s),v) - F(X*(s),u*(s)) \' ds 




(4.4) 



But Taylor expansion implies the three identities: 



F{X,{s),v)-F{X*{s),v) 

= / F,{X*{s),u*is) + \iX,{s)-X*{s)))iX,{s)-X*{s))dX, (4.5) 
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FiX*{s),v)-F{X*{s),u*{s)) 

= [ F,{X*{s),u*is) + Xiv-u*is)))iv-u*is))dX, (4.6) 
Jo 

and 

G(X,(s))-G(X*(s)) 

= j G,{X*{s) + \{X,{s)-X*{s))){X,is)-X*{s))d\ 

=: (G LQiK)). (4.7) 

Then, by using (14.71) . the isometry property (12. ip . (12. 2p and (El) we deduce 
that for all t e [to, to + 

E[| r(G(X,(.))-G(X*(.)))dM(.)H=E[| [\{s)dM{s)\'] 

Jto Jto 

= E[ rii$(.)Q^/^(.)ii^c^.] 

Jto 

< E[ rii<f(3)iii,(K)rf^] 

= E[f\\ [ G^{X*is) + XiX,{s)-X*{s))){X,is)-X*is))dX\\l^^^^ds] 

Jto Jo 

< E[ f f\\G^{X\s) + \{XM-X\s))){XM-X\s))\\l^^^^d\ds] 

Jto Jo 

< C2E[ f\X,{s)~X*{s)\'ds]. (4.8) 

Jto 

Therefore, from (jMl), (jMl), (El) and ^B>, it follows evidently that 

E [ \X,{t) - X*(t) |2 ] < 3 (Ci (t - to) + C2) r E [ I X,(s) - X*(s) p ] ds 

Jto 

+ 3{t-to)C3 f E[\v-u*{s)\^]ds, 

Jto 

for all t G [to, to + £]• 

Hence by using Gronwall's inequality we obtain 

E[|X,(t) - X*(t) n < 3C3(t-to)e^('^^(*-*'')+^')(*~*°) 

rto+s 

X / E[\v-u*{s)\^]ds, (4.9) 
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for all t G [to, to + £]■ Consequently, 



E 



rto+£ 

/ \Xe{t)-X*{t) \^dt] <3C3e^e^^^''+^^^' 

J to 

rto+e 

X / ¥.[\v -u*{s)\^]ds. (4.10) 

Jtn 



It follows then from (I4.4p . (14. 9p . standard martingale inequalities, ( 14. 8p and 
fICTD that 

E[ sup |X,(t)-X*(t)H 

rto+e 

< 3C3[3(Cie + 4C2)£e^(^^"+^')" + 1]£ / ¥.[\v - u*{s)\'']ds. (4.11) 

J to 

Next, for + e <t <T, we have 

X,{t) - X*(t) = X,(to + ^^) - X*(to + £) 

+ / [F{X,{s)X{s))-F{X*{s),u*{s))]ds 

Jto+e 

+ / [G{X,{s))-G{X\s))]dM{s). (4.12) 



Thus by working as before and applying (14.91) we derive 

r-T i-to+e 



E 



pi rto+e 

[ \X,{t) ~X*{t)\^dt]<9C3 6^e^^^'^ E[\v-u*{s)\^]ds 

Jto+e J to 



and 



E[ sup \X,{t)-X*{t)\^]<27C3ee^^^''>[l + {{T-to-e)Ci + 4C2)e' 

to+e<t<T 

"to+e 

X 



rto+e 

/ E[\v-u*{s)\'^]ds, (4.13) 

J to 



where ^(e) = [3 + 3 (T - to - e)^] Ci + {T -to + 2e) C2. 
Now (l4TTi) and (l4J3i) imply that 

E[ sup |X,(t)-X*(t)H < (C5(£) + C6(£)) / E[|t;-M*(s)H(is 

to<i<T Jto 

with the constants 

^5(5) = 3 [ 3 (Ci e + 4C2) £ + l]e 
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and 

Ce (e) = 27C,e e'^^(^) [1 + {{T - to - e) C, + AC^) e]. 
This completes the proof. ■ 

Remark 4.3 We note that for a.e. s, 



s+e 



E[\(j){X*{t),u*{t))-(l){X*{s),u*{s))\']dt^O, ase^O, (4.14) 



for (j) = F,i. Indeed, if for example, (p = F, then we may argue as in to 
see that 



E [ \F{X*{t),u*{t)) - F{X*{s),u*{s))\' ] dt 

^ E[| / F^{X*{t),u*{s) + \{u*{t)-u*{s))){u*{t)~u*{s))dX\^dt 







s+e 



<- I E[\u*it) -u*is)\'^]dt. (4.15) 

^ J s 

But since /g^E [ \u*{t) — u*(s)p] dt < oo (for fixed s), then, as it is well-known 
from measure theory (e.g. fTW). there exists a subset O of [0,T] such that 
P(0) = 1 and the mapping O 9 t i— > E [ \u*{t) — u*{s)\'^] is continuous. Thus, 
if s & O, this function is continuous in a neighborhood of s, and so we have 

1 

- / E[\u*{t)-u*{s)\'^]dt^O, ase^O, 

^ J s 

which by U.15\ ) implies ^.14] ) for cj) = F. 

We will choose to such that fl4.14p holds for (j) = F,i. This assumption will 
be considered until the end of Section [51 

Lemma 4.4 Assume (El). Let 

Ut) = I (x,(t) -x*(t)) -p(t), t e [to,T]. 



Then 



liniE[|^,(T)n = 0. 

e—^0 
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Proof. First note that, for to ^ ^ ^ + 



dUt) = ^[F{X,{t),v) - F{X*{t),u*{t)) - eF,{X*{t),u*{t))p{t)]dt 
+ ^ [G{X,it)) - GiX*{t)) - sG,iX*{t))p{t)]dM{t), 



Uto) = -{F{X*{to),v) - F{X*{to),u*ito))). 



Thus 



1 rto+e 

Uto + e) = - / [F{X,{s),v)-F{X*{s),v)]ds 

£ Jto 

+ - [ '^\f{X*{s),v) - F{X*{to),v) ] ds 

^ J to 



1 



to 
to+e 



e 



+ - / [F{X*{t,),u\t,))-F{X\s),u\s))]ds 



to 

+ - r^\G{XM)-G{X*{s))]dM{s) 

£ Jto 

to+e 

F, {X*{s),u*{s))p{s) ds 

to 
to+e 

G, {X*{s))p{s)dM{s). 

to 



By using ([211]), (O and (El) we deduce 



E[\Uto + e)\']<QCiE[ sup \X,{t) - X*{t)\^] 

to<t<to+e 

+ 6 sup E[\F{X*{t),v)-F{X*{to),v)\^] 

to<t<to+e 

+ - E [ \F{X*{s), u*{s)) - F{X*{to),u*{to))\' ] ds 

^ Jto 

+ ^E[ sup \X,{t)-X*it)\^] 

£ to<t<to+e 

+ 6{Gi + G2)E[ ["^ \p{s)\Us]. (4.16) 

Jto 
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But from fICTD 
-E[ sup |X,(t)-X*(t)H 

£^ to <*<*()+£ 

/•to+e 

<3C3[3(Ci£ + 4C2)£e3(^^'+^^)^ + 1] / E [ |t; - 

J to 

-> as £ 0. (4.17) 

Also as in fl4.5p . by applying (El) and fl4.9p . one gets 
E[\F{X*{t),v)-F{X*{to),v)\'] 

= E[| / F,.(X*(to) + A(X*(t)-X*(to)),t;)(X*(t)-X*(to))rfA|2 
Jo 

< C^E[\X*it)-X*ito)\'] 

<3CiC3ee^i^''^^')' [' E[\v - u*{s)\^]ds ^ as £ ^ 0. (4.18) 

J to 

Thus, by applying Lemma |li2l fl4.18p . fl4.17p . (14.141) and Lemma |0 in 
(I4.16p . we deduce 

E[|^e(to + ^) T] -> ase^O. (4.19) 
Let us now assume that + e < t < T. In this case we have 

dUt) = ^[FiX,it),u*it))-FiX*{t),u*it))-6F.,iX*it),u*it))pit)]dt 

+ ^[G{X,it))~GiX*{t))-sG,iX*{t))p{t)]dM{t), 
or, in particular, by setting 

<l,(s) = f[G^{X*{s) + X{X,{s) - X\s))) - G^{X\s)) ] p{s) dX, 
Jo 

we get 

Ut)=Uto + e)+ [ [ F,{X*{s) + \{X,{s)~X*{s)),u*{s))Us)dXds 

Jto+£ Jo 

+ f f G,,{X*{s) + \{X,{s)-X*{s)))Us)d\dM{s) 

Jto+e Jo 

+ f [\f,{X*{s) + X{X,{s)-X*{s)),u*{s)) 

Jto+e Jo 

-F,{X*{s),u*{s))]p{s) dXds 
+ / ^,{s)dM{s), 

Jto+£ 
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for all t E [to + e,T]. Hence by making use of the isometry property (12. ip it 
holds Vt G [to + e,T], 



ds 



+ 5E ' 



E[|e.(t)H<5E[|e.(to + ^)r] + 5(Ci + C2) / E[\Us 

J to+e 

{F,iX*{s) + XiX,is)-X*is)),u*is)) 

-F^{X*{s),u*{s)))p{s) dX ds 
+ 5E[/ \\^,{s)Q'/\s)\\lds]. (4.20) 

Jtn 



to JO 



But as done for the second equality and first inequality in (14.81) we can derive 
easily that 



T 

nl \ms)Q'/\s)\\lds\ 

Jtn 



= E 

< E 
= E 

< E 



to 
t 

\\^eis)\\l^(K) ds] 

to 

t rl 



to 
t rl 



[ II / [G,{X*{s) + X{X,{s)-X*{s)))-G,{X*{s)Ms)dX\\l^^^^ds] 

Jta JO 

G,{X*{s) + X{X,{s)-X*{s))) 

~G^iX*{s))]p{s)\\l^^^^dXds]. (4.21) 



to JO 



Therefore, from Lemma |4.2[ the continuity and boundedness of G^ in (El), 
Lemma 14.11 and the dominated convergence theorem to get that the last term 
in the right hand side of (14.211) goes to as e — >■ 0. 

Similarly, the third term in the right hand side of (I4.20p converges also to 
as £ 0. 

Finally, by applying Gronwall's inequality to (I4.20p . and using (I4.19p - (l4.2ip . 
we deduce that 

sup E[|^e(t) T] as e ^0, 

to+£<t<T 

which proves the lemma. ■ 
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Lemma 4.5 Assume (El) and (E2). Let ( be the solution of the equation: 



dCit) = UX*{t),u*{t))p{t)dt, to<t<T, 
ato) = i{X*ito),v) - iiX*ito),u*ito)). 



Then 



lim E 

e-s-O L ' e 



{i{x,{t), u,{t)) - e{x*{t),u*{t)))dt - at) 



to 



Proof. Let 



Veit) = - / {iiX,it),uM)-iiX*it),u*it)))dt-CiT), 



to 

fort G [to, T]. Then r^, (to) = -{i{X*{to),v) — £(X*(to),M*(to))). So one can pro- 
ceed easily as done in the proof of Lemma [4.41 to show that E [ | f]e{T) — )■ 0, 
though this case is rather simpler. ■ 

Let us now for a mapping : K ^ M. denote by V\I' to the gradient 
of which is defined, by using the directional derivative D'^{x){k) of \1/ at a 
point X e K in the direction of k e K, as (V\['(x), k) = D'^{x){k) (= "^xik)). 
We shall sometimes write Vx"^ for V\['(x). 

Corollary 4.6 Under the assumptions of Lemma \4.5\ 

^ JM))l^, = E [ (V h{X*{T)), p{T)) + C(T) ]. (4.22) 

Proof. Note that from the definition of the cost functional in ( 13. 2p we see 

that 



1 



E 



+ 



/i(x,(T))-/i(x*(r)) 



{iiX,is),uM)-(iX*is),u*is)))ds 



to 



E 



' hx{X*{T) + A(X,(T) - X*{T))) i^^I^ — X*{T)) 



+ 



{i{X,{s),Ue{s)) - i{X*{s),U*{s)))ds 



to 



Now let £ — > and use the properties of h in (El), Lemma K4l and Lemma 1431 
to get (02]) . ■ 
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5 Maximum principle 

The maximum principle is a good tool for studying the optimality of controlled 
SDEs like ( ]3.ip since in fact the dynamic programming approach for similar 
optimal control problems require usually a Markov property to be satisfied by 
the solution of (13. ip . cf. for instance [361 Chapter 4]. But this property does 
not hold in general especially when the driving noise is a martingale. 

Let us recall the SDE (13. ip and the mappings in (13. 2p . and define the 
Hamiltonian H : [0,T] x x K x O x K x L2{K) — )■ M for (t, uj, x, u, y, z) G 
[{),T]xVLx K xO X K X L2{K) by 

H{t, CO, X, u, y, z) ■= iix, u) + {F{x, u),y) + (^(x) Q'/\t, to) , z)^ . (5.1) 

The adjoint equation of (13. ID is the following BSDE: 

-dY<\t) = y^H{t,X<-\t),u{t),Y<-\t),Z<\t)Q^I'^{t))dt 

- Z<\t) dM{t) - dN<-\t), to<t<T, (5.2) 
F«(-)(T) = V/i(X"(-)(T)). 

The following theorem gives the solution to BSDE (15. 2p 
in the sense that there exists a triple (Y^^'\ Z^^'\ N"^^'^) in 
L^O,T;K) X A^{K;V,M) X M^^'^j.^iK) such that the following equality 
holds a.s. for all t e [0,T], N{0) = and is VSO to M: 

Y<\t) = ^ + j\,H{s,X<\s),u{s),Y''^\s),Z<\s)Q'/\s))ds 
Z''^-\s)dM{s) - [ rfAr"(-)(s). 



Theorem 5.1 Assume that (E1)-(E2) hold. Then there exists a unique solu- 
tion {Y<\Z<\N<-'^) of the BSDE (EJ). 

For the proof of this theorem one can see [2] . 

We shall denote briefly the solution of (15.20 . which corresponds to the 
optimal control u*{-) by {Y* , Z* ,N*). 

In the following lemma we shall try to compute E [ (Y*{T), p(T)'^ ]. 
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Lemma 5.2 

E [ ( F*(T),p(T) > ] = - E [ r UX*{s),u*{s))pis) ds ] 

J to 

+ E [ (F*(to), FiX*{to),v) - FiX*ito),u*{to)) ] . (5.3) 



Proof. Use Ito's formula together to compute d (Y* (t) , p(t)'^ for t G [to)^]; 
and use the facts that 

r{p{s) , V.H{s, X*{s),u*{s), Y*{s), Z*{s)Q'/'is))) ds 

Jtn 

UX\s),u\s))p{s) + {R,{X\s),u\s))p{s) ,Y\s) >1 ds 
+ f { G.,{X*is))p{s)Q'/\s) , Z*{s)Q'/\s) >2 ds, 



to 



which is easily seen from (15. ip . ■ 

Now we state our main result of this section. 

Theorem 5.3 Suppose (E1)-(E2). If {X*,u*{-)) is an optimal pair for the 
problem ( 1 5*. l\) - l[37^) . then there exists a unique solution {¥*, Z* , N*) to the 
corresponding BSDE such that the following inequality holds: 

Hit,X*it),v,Y*{t),Z*{t)Q'/\t)) 

> H{t,X*{t)X{t),Y*{t),Z*{t)Q^'\t)) (5.4) 

a.e.te [0,r], a.s. Vf G U. 

Proof. We note that since «*(■) is optimal, ^ J{ue{-))\e=o > 0, which implies 
by using Corollary 14.61 that 

E[(r*(T),KT)>+C(T)]>0. (5.5) 

On other hand by applying (15.51) and Lemma 15.21 one sees that 

< - E[ / t,{X*{s),u*{s))p{s)ds] 

J to 

+ E [ (r*(to), F{X*{t^),v) - FiX*ito),u*ito)) + C(T) ]. (5.6) 

But ^ 

C(T) = C(to)+ f UX*{s),u*{s))p{s)ds 



to 
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and 

Hito,X*ito),v,Y*ito),Z*ito)Q'/'ito)) 

-H{to,X*{to),u*ito),Y*{to),Z*{to)Q'/\to)) 
= C(to) + {Y*{to), F{X*{to), v) - F{X*{to), u*{to))). 
Hence fl5.6p becomes 

< E [H{to,X*{to),v,Y*{to),Z*{to)Q'/\to)) 

- H{to, X*{to),u*{to), y*(to), Z*ito)Q'/\to))]. (5.7) 

Now varying to as in (14.141) shows that (15. 7p holds for a.e. t., and so by 
arguing for instance as in [10^ P. 19] we obtain easily (15. 4p . ■ 

Remark 5.4 Let us assume for example that the space K in Theorem \5.1\ is 
the real space M and M is the martingale given by the formula 



M{t) = [ a{s)dB{s), t G [0,T], 
Jo 



'0 

for some a G Ljr{0,T;M.) and a one dimensional Brownian motion B. If 
a{s) > for each s, then J^t{M) = J^t{B) for each t, where 

J^^[R) = (j{R{s),0< s<t} 

forR = M, B. Consequently, by applying the unique representation property for 
martingales with respect to {J-t(M), t > 0} or larger filtration in JE, Theorem 
2.2] or 1^ and the Brownian martingale representation theorem as e.g. in [I4, 
Theorem 3.4, P. 200], we deduce that the martingale A^"(') in /i5. 2\) vanishes 
almost surely if the filtration furnished for the SDE liS. 1\) is {J-i(M), < t < 
T}. This result follows from the construction of the solution of the BSDE h5.2^] . 
More details on this matter can be found in [2, Section 3]. As a result, in this 
particular case BSDE ( 15. 2^ fits well with those BSDEs studied by Pardoux & 
Peng in [2^ , but with the variable aZ replacing Z there. 

Thus in particular we conclude that many of the applications of BSDEs, 
which were studied in the literature, to both stochastic optimal control and 
finance (e.g. f3l\] and the references therein) can be applied directly or after 
slight modification to work here for BSDEs driven by martingales. For example 
we refer the reader to f2^ for financial application. Another interesting case 
can be found in JM]- 

On the other hand, in this respect we shall present an example (see Exam- 
ple \6.4\ ) in Section\^ by modifying an interesting example due to Bensoussan 

n 
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6 Sufficient conditions for optimality 

In the previous two sections we derived Pontyagin's maximum principle which 
gives necessary conditions for optimahty for the control problem fl3.ip -f lX^ . 
In the following theorem, if we have also a convexity assumption on the control 
domain U, we shall obtain sufficient conditions for optimality of this optimal 
control problem. This concerned result is a variation of Theorem 4.2 in [3]. 

Theorem 6.1 Assume (El) and, for a given u*{-) G Uad, X* o,nd 

(Y*,Z*,N*) be the corresponding solutions of equations /13. 1\) and /15.S\) re- 
spectively. Suppose that the following conditions hold: 
{{) U is a convex domain in O, h is convex, 



(ii) {x,v) ^ H{t,x,v,Y*{t),Z*{t)Q'/\t)) is convex for all t G [0, T] a.s., 
(zn) H{t, X*{t), n*(t), F*(t), Z*it)Q'/'it)) 



for a.e. t E [0, T] a.s. 

Then (X*, «*(■)) is an optimal pair for the control problem liS. ip -z TOj) . 

Proof. Let u{-) E Uad- Consider the following definitions: 



min H{t,X*{t),v,Y*{t),Z*{t)Q 




and 



h :=E [/i(X*(r)) -/i(X"(-)(T))]. 



Then readily 



J{u*{-))-J{u{-)) = h + h. 



(6.1) 



Let us define 




H{t,X-^-\t)Mt),Y*{t),Z*{t)Q}/\t)))dtl 



/4:=E[ / {F{X*{t)X{t))-F{X<-\t),u{t)),Y*{t))dtl 



/5:=E[ / {{G{X''^*^-\t))-G{X<\t)))Q^'\t),Z*{t)Q^'\t))^dtl 
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and 



h :=E 



T 







{V,H{t,X*{t),u*{t),Y*{t),Z^*^\t)Q'/\t)), 

X*(t) -x"(-)(t))rft 

From the definition of H in (15. ip we get 

h = h-h- h. (6.2) 
On the other hand, from the convexity of h in condition (ii) it follows 
h{X*{T))-h{X<-\T)) < {Vh{X*{T)) ,X*{T) - X<\T)) a.s., 
which implies that 

/2 < E [( F*(T) ,X*(T) -X"(T) )]. (6.3) 

Next by applying Ito's formula to compute d {Y*{t) , X*(t) — X"( )(t)) and 
using equations (15. 2 p and (13. ip we find with the help of ( 16. 3p that 

h<h + h-h. (6.4) 

Consequently, by considering (16.11) . (16.21) and (16. 4p it follows that 

J{u*{-))-J{u{-))<h~h. (6.5) 

On the other hand, from the convexity property of the mapping 
{x,v) ^ H{t,x,u,Y*{t), Z*{t)Q}^'^{t)) in assumption (iii) the following in- 
equality holds a.s.: 

T 

H{t,X*{t),u*{t),Y*{t),Z\t)Q}/\t)) 







< 



- H{t,X<\t)Mt).y*{t).Z*{t)Q}l\t))) dt 

r {V.H{t,X*{t)X{t),Y*{t),Z*{t)Q}'\t)) , 
Jo 

x*(t) -x"(-)(t) ) dt 

{ VuH{t,X*{t)X{t),Y*{t),Z*{t)Q}/\t)),u*{t)-u{t))^dt. 
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As a result 

h<h + 17, (6.6) 

where 

I, = e\ f {VuH{t,X*{t),u*{t),Y*{t),Z*{t)Q}/\t)) , 
Jo 

u*{t)-u{t) )^dt 

Since v H{t,X*{t),v,Y*{t),Z*{t)Q}/'^{t)) is minimum aX v = u*{t) by the 
minimum condition (iii), we have 

( VuH{t,X*{t),u*{t),Y*{t),Z*{t)Q'l\t)) ,u*{t) - u{t) )^ < 0. 

Therefore /7 < 0, which by fl6.6p imphes that /s — /e < 0. So (16.51) becomes 

JK(-))-^K-))<o. 

Now since m(-) G is arbitrary, this inequahty proves that (X*, «*(•)) is 
an optimal pair for the control problem fl3.ip - fl3.3l) as required. ■ 

Example 6.2 Let m he a continuous square integrable one dimensional mar- 
tingale with respect to {J't}t such that < m >t = J^;a{s)ds y <t <T for 
some continuous a : [0,T] — )■ (0, 00). Consider M{t) = (3m{t)(= J^^ (3 dm{s)) , 
with 13 ^ being a fixed element of K. Then M e M'^'''{K) and « M »t 
equals [3 ® (3 j^a{s)ds, where [3 ® [3 is the identification of [3 ® [3 in Li(K), 
that IS (f®j){k) = {/3,k)l3, k e K. Also < M >t = \/3\'^ a{s) ds. Now 
letting Q{t) = (3 ® (3 a{t) yields that « M »t = jlQ{.s)ds. This process 
Q{-) is hounded since Qit) < Q W t, where Q = B ® (3 max aft). It is also 

V / 0<t<T 

easy to see that Q}''^{t){k) = ^^^^ a^/^{t). In particular (3 G Q^/^{t){K). 

Let K = L^(]R"). Let M he the ahove martingale. Suppose that O = K. 
Assume that G G Lq{K) or even a hounded linear operator from K into itself, 
and F is a hounded linear operator from O into K. Let us consider the SDE: 

dX{t) = Fu{t) dt+ {X{t) ,/3) G dM{t), t G [0,T], 
X(0) = xoeK. 

For a given fixed element c of K we assume that the cost functional is given 
hy the formula: 

JH-)) = E[ r\uit)\'dt]+E[{c,XiT))], 
Jo 
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and the value function is 

r = inf {J{u{-)) : u{-) G Uad}- 

This control problem can be related to the control problem liS. 1\) - I[37S\) as 
follows. We define 

F{x, u) = F u, G{x) = {x , (3) G, i{x, u) = and h{x) = (c , x) , 

where (x, u) E K x O. 

The Hamiltonian then becomes the mapping 

H : [0,T] X n X K X O X K X L2{K) M, 

H{t, X, u, y, z) = \u\^ + {Fu,y) + {x,P) {G Q}'\t) , z)^ , 

{t,x,u,y,z) e K X O X K X L2{K). 

It is obvious that H{-, y, z) is convex with respect to (x, u) for each y and 
z and S7xH{t, x, m, y, z) = (G Q^/^{t) , z) (3. 

Next we consider the adjoint BSDE: 

r - dY{t) = [ {G Q^/\t) , Z{t))^ 13] dt- Z{t) dM(t) - dN{t), 

I yiT) = c. 

This BSDE attains an explicit solution Y{t) = c, since c is non-random. But 
this implies that Z{t) = and N{t) = for each t G [0,T]. 

On the other hand, we note that the function O 3 u ^ H{t, x, u, y,z) 
attains its minimum at u = —^F*y, for fixed {x,y,z). So we choose our 
candidate for an optimal control as 

u*{t, oo) = -^F* Y{t, oo) = -^F*c {eU:=0), 

With this choice all the requirements in Theorem \ 6.1\ are verified. Con- 
sequently u*{-) is an optimal control of this control problem with an optimal 
solution X given by the solution of the following closed loop equation: 

r dX{t) = -\FF* Y{t) dt + , /3) G dM{t), 

{ X{0) =XoeK. 

The value function takes the following value: 

J* = ^ \F*c\^T + E [ (c ,X(T)) ]. 
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Remark 6.3 It would be possible if we take h{x) = x ^ K, in the pre- 
ceding example and proceeds as above. However if a result of existence and 
uniqueness os solutions to what we may call 'forward-backward stochastic dif- 
ferential equations with martingale noise" holds, it should certainly be very 
useful to deal with both this particular case and similar problems. 

Example 6.4 Let O = K. We are interested in the following linear quadratic 
example, which is gleaned from Bensoussan [10, P. 33]. Namely, we consider 
the SDE: 

dX{t) = {A{t)X{t) + C{t)u{t) + f{t)) dt + {B{t)X{t) + D{t)) dM{t), 
X(0) = Xo, ^^-^^ 

where B{t)x = {-f{t) , x) G{t) and A,-f,C : [0,T] x K ^ K, f : [0,T] 
K, G,D : [0,T] —7- Lq{K) are measurable and bounded mappings. 

Let P,Q : [0,T] X K K, Pi : K ^ K be measurable and bounded 
mappings. Assume that P, Pi are symmetric non-negative definite, and Q is 
a symmetric positive definite and Q~^{t) is bounded. For SDE ( [g. 7\ ) we shall 
assume that the cost functional is 

JH-)) =^[£(1 (^W^"" W , X<\t)) + \ {Q{t)u{t) , u{t)) ) dt 

+ ^(PiX""(T),X<)(T)>], (6.8) 

foru{-) e Uad- 

The control problem now is to minimize \6.^) over the set Uad O'nd get an 
optimal control u*{-) G Uad-, that is 

J{u\-)) = inf{J(n(-)) : u{-) G Uad]- (6.9) 

By recalling Remark \3.1\ we can consider this control problem ( 6. 7 )- f67^} 
as a control problem of the type h3. l\) - [373^} . To this end, we let 

Fit, X, u) = A{t)x + C{t)u + fit), 

Git,x) = {j{t),x)Git) + Dit), 

iit, x,u) = ^ {P{t)x , x) + i {Q{t)u , u), 

h{x) = i {Pix,x). 
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Then the Hamiltonian H : [0,T] x x K x K x K x L2{K) R is given by 
Hit, X, u, y, z) = i{t, X, u) + {F{t, x, u) ,y) + {Git, x) Q^/^it) , z)^ 

+ {Ait)x + Cit)u + fit),y) 

+ {i{^it),x)Git) + Dit))Q'/\t),z)^. 

We can compute VxH directly to find that 

W.Hit, X, u, y, z) = Pit)u + A*it)x + {Git)Q}'\t) , z)^ -fit). 

Hence the adjoint equation of ^6. 7] ) takes the following shape: 

' - dY<-\t) = + Pit)X<-\t) 

+{Git)Q^'\t) , Z<\t)Q'/\t))^jit) ) dt 

-Z<-\t)dMit) - dN^'^-^t), 

r«(-)(T) = PiX"(-)(T). 

Now the maximum principle theorems (Theorem \5.3\. Theorem \6. 1\) in this 
case hold readily if we consider Remark \3.1\ again, and yield eventually 

G*it)Y*it) + Qit)u*it) = 0. 

Acknowledgement. This author would like to thank the Mathematics 
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